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(54) Disk array controller and method for adapting the number of control units in a disic array 
controller 



(57) A disk array controller is made up of multiple 
disk an-ay control units (1-1 , 1-2, 1-3) for implementing 
tlie data read/write operation and each having channel 
IF units (11 ), disk IF units (12), cache memory units (14) 
and shared memory units (1 3). The disk array controller 
further includes Interconnections (24. 25, 210, 220) for 



interconnecting the shared memory units (13) and inter- 
connecting the cache memory units (1 4) across the bor- 
der of disk array control units. Thereby alleviating the 
deterioration of perfomnance due to the data transfer be- 
tween the disk an-ay control units, when the multiple disk 
array control units are to be operated as a single disk 
array controller. 
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Description 

BACKGROUND OF THE INVENTION 

[0001] The present invention relates to a controller of 
a disk array apparatus which stores data on multiple 
hard disk drives. 

[0002] As compared with the computer main storage 
which is based on the semiconductor memory, the disk 
subsystem (will be termed simply "subsystem") which is 
based on the magnetic disk memory Is sluggish in re- 
gard to the I/O performance by a degree of the order of 
3-4 digits, and continuous efforts have been paid to re- 
duce the difference of these memories, i.e., improve the 
I/O performance of the subsystem. 
[0003] A known scheme to improve the I/O perform- 
ance of the subsystem is the adoption, of a disk array 
system In which multiple hard disk drives are organized 
to fonri a subsystem, and data is stored on the multiple 
hard disk drives. 

[0004] For example, a prior art system shown in Fig. 
2 Includes multiple channel IF units 11 which implement 
the data transfer between host computers 50 and disk 
array controllers 2, multiple disk IF units 12 which im- 
plement the data transfer between magnetic disk drives 
5 and disk array controllers 2, cache memory units 14 
which temporarily store data of the hard disk drives 5, 
and shared memory units 13 which store control infor- 
mation for the disk array controllers 2 (e.g., information 
on data transfer control between the channel IF units 1 1 
and disk I F units 1 2 and the cache memory units 1 4 and 
control infonnation for data stored in the hard disk drives 
5). In each disk array controller 2, the shared memory 
unit 13 and cache memory unit 14 can be accessed by 
all channel IF units 11 and disk IF units 12. 
[0005] In the disk array controller 2, the channel IF 
units 1 1 and disk IF units 1 2 are connected to the shared 
memory unit 1 3 by an interconnection 21 , and the chan- 
nel IF units 11 and disk IF units 12 are connected to the 
cache memory unit 14 by another interconnection 22. 
[0006] The channel IF unit 11 has the interface for the 
connection to host computers 50 and a microprocessor 
(not shown) which controls the data transaction with the 
host computers 50. The disk IF unit 12 has the interface 
for the connection to hard disk drives 5 and a microproc- 
essor (not shown) which controls the data transaction 
with the hard disk drives 5. The disk IF units 12 have the 
RAID function. 

[0007] This conventional disk an'ay controller 2 has 
an upper limit of disk storage capacity which can be con- 
nected to it, and therefore for storing data in excess of 
the volume of data manageable by one disk array con- 
troller 2, multiple disk array controllers 2 are installed 
and channels from the host computers 50 are connected 
to the disk array controllers 2. Furthermore, for connect- 
ing host computers 60 in excess of the number of host 
channels which can be connected to one disk array con- 
troller 2, multiple disk array controllers 2 are installed 



and connected to the host computers 50 individually 
[0008] For data transfer between two disk array con- 
trollers 2, channels from the host computers 50 are con- 
nected to the two disk array controllers 2 and data is 
5 transferred via the host computers 50. 

[0009] Another prior art system shown in Fig. 3, which 
is disclosed in U.S. Patent No.5,680,640, is designed for 
example to transfer data between two disk array con- 
trollers 3 based on the connection of part (two lines in 

10 Fig. 3) of the computer interface paths of the disk array 
controllers 3 and the data transfer from a hard disk drive 
5 which is connected to one disk array controller 3 to 
another hard disk drive 5 which is connected to another 
disk array controller 3 through the data transfer path 8. 

15 [001 0] Another prior art system shown in Fig. 4 is de- 
signed for example to store data in excess of the volume 
of data manageable by one disk array controller, con- 
nect host computers in excess of the number of host 
channels which can be connected to one disk array con- 

20 trolier, or transfer data among multiple disk array con- 
trollers, as shown in Fig. 4, based on the installation of 
multiple disk array controllers 4 and the connection of 
their computer Interface paths to the host computers 50 
via a switch-based interconnection 23. 

25 [0011] A disk array processing device disclosed in 
Japanese Unexamined Patent Publication No.Hei- 
1 1 -66693 includes two director units which use a shared 
memory to recover data when a data spindle which 
fomns a disk array runs out of control. This patent pub- 

30 lication, however, does not show the installation of mul- 
tiple disk arrays. 

[0012] In large business enterprises which typically 
include banks, stock traders, and telephone companies, 
there are trends of cutting the expenditure for the oper- 
as ation, maintenance and management of computer sys- 
tems and storage systems based on the centralized in- 
stallation of computers and storages which have been 
Installed distributively in many places. In this movement 
with the times, large high-end disk array controllers are 
40 required to bearthe channel interface for the connection 
to several-hundreds or more host computers and huge 
storage capacities of several-hundreds terabytes or 
more. 

[001 3] At the same time, due to growing open markets 
45 in recent years and storage area networks (SANs) which 
are expected to prevail in coming years, there are rising 
demands of small-scale (compact) disk array controllers 
which are comparable in performance and reliability with 
large high-end disk array controllers. 
50 [0014] The former requirement will conceivably be 
met by connecting multiple conventional large high-end 
disk array controllers to build a huge disk array control- 
ler. The latter requirement will conceivably be met by 
arranging a lowest-level model (e.g., with small num- 
S5 bers of channel IF units and disk IF units) of a conven- 
tional large high-end disk array controller. An additional 
conceivable scheme is to connect a plurality of this com- 
pact disk array controller thereby to build a controller 
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which supports systems of medium to large scales 
which have been supported by a conventional disk array 
controller. 

[001 5] Accordingly, it becomes necessary for disk ar- 
ray controllers to have the scalability so as to be con- 
structed to cover the range from a small to huge-scale 
controllers based on the same basic architecture of high 
performance and high reliability, and therefore there 
arises a demand of a disk array controller, a plurality of 
which are integrated to operate as a single disk array 
controHer 

[001 6] However, although the prior art system shown 
in Fig. 2 can have an increased number of channels and 
increased storage capability for the host computers 50 
by simply increasing the number of disk array controllers 
2, the host computers 50 need to connect channels to 
ail disk array controllers 2 so that one host computer 50 
can transact data with multiple disk array controllers 2. 
Moreover, it is necessary for one host computer 50 
which is going to make access to data to identify a disk 
array controller 2 that is connected to the hard disk drive 
5 which stores the data. On this account, it is difficult to 
operate multiple disk array controllers as a single disk 
array controller. 

[0017] The prior art system shown in Fig. 3 has disk 
array controllers 3 interconnected by the data transfer 
path 8, enabling one host computer 50 which Is connect- 
ed to a certain disk array controller 3 to make access to 
data on a hard disk drive 5 which is connected to other 
disk array controller 3, and accordingiy it is possible to 
operate multiple disk array controllers 3 as a single disk 
array controller. 

[001 8] However, in case a host computer 50 issues a 
data read request to a disk array controller 3 and the 
data is absent on the hard disk drives 5 connected to 
the controller 3, it is necessary for the disk array con- 
troller 3 to send the read request to other disk array con- 
troller 3 over the data transfer path 8, receive the re- 
quested data over the data transfer path 8 from the disk 
array controller 3 which is connected to the hard disk 
drive 5 on which the data is stored, and send the re- 
quested data to the host computer 50. On this account, 
this system suffers a degraded perfonmance when a 
host computer 50 makes access to data which is stored 
on a hard disk drive 5 connected to a disk array control- 
ler 3 other than the disk array controller 3 connected to 
itself. 

[001 9] For coping with this matter, if a host computer 
50 transfers in advance data of frequent access, which 
is stored on a hard disk drive of a disk array controller 
other than that connected to itself, to the hard disk drive 
5 of the disk array controller connected to itself, the sys- 
tem also suffers a degraded performance due to the da- 
ta transfer over the data transfer path 8. 
[0020] In the prior art system shown in Fig. 4, each 
host computer 50 can access to all disk array controllers 
4 via the switch-based interconnection 23. However, in 
order to operate multiple disk array controllers 4 as a 



4 

single disk array controller, each switch of the intercon- 
nection 23 must have a map indicating as to which of all 
disk array controllers 4 connected to the switch stores 
data, and must have a function of analyzing the access 

5 request from a host computer 50 and designating a disk 
array controller 4 which stores the requested data. 
[0021 ] Due to the need of request command analysis 
by the switches besides the command analysis by the 
conventional channel IF units 11, this system suffers a 

10 degraded performance in contrast to the direct connec- 
tion of the host computers 50 to the disk array controllers 
4. 

[0022] The high-end disk array controllers have fol- 
lowing functions. 
15 [0023] As a specific operational function, a data set 
(corresponding to a logical volume) for one work is du- 
plicated and stored and both the original and duplicate 
data sets are updated simultaneously in the ordinary 
work. At the emergence of the need of backup for the 
data set for example, updating of the duplicate data set 
is suspended and it is used for the backup, while the 
original data set is used continuously for the work, and 
the original and duplicate data sets are rendered the 
matching process on completion of backup. 
[0024] For accomplishing this operational function, 
with duplicate data sets being held in different disk array 
controllers, the prior art systems shown in Fig. 2 to Fig. 
4 all need to transfer data sets among the disk array 
controllers, resulting in a significantly deteriorated sys- 
tem performance. 

SUMMARY OF THE INVENTION 

[0025] An object of the present invention is to provide 
a disk array controller having the scalability so as to be 
constructed to cover the range from a small to huge- 
scale controllers based on the same basic architecture 
of high performance and high reliability. 
[0026] More specifically, the present invention is in- 
tended to provide a disk array system which is capable 
of alleviating the deterioration of performance caused 
by the data transfer among multiple disk array control- 
lers which are designed to operated as a single disk ar- 
ray controller, and accomplish the function of a disk ar- 
ray controller based on a plurality of disk array control- 
lers while alleviating the deterioration of performance. 
[0027] In order to achieve the above objectives, the 
present invention resides in a disk array controller which 
includes a plurality of disk array control units, each hav- 
ing one or more channel interface units for interfacing 
with a computer, one or more disk interface units for in- 
terfacing with disk drives, a cache memory unit which is 
connected to the channel interface unit and disk inter- 
face unit and adapted to store temporarily data which is 
written to or read out of the disk drives, and a shared 
memory unit which Is connected to the channel interface 
unit and disk interface unit and adapted to store control 
infomnation of data transfer between the channel inter- 
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face unit and disk Interface unit and the cache memory 
unit and control information for the disk drives, and hav- 
ing a disk control function for Implementing the data 
read/write operation in response to a data read/write re- 
quest from the host computer by operating on the chan- s 
nel interface unit to transfer data between the interface 
with the host computer and the cache memory unit and 
operating on the disk interface unit to transfer data be- 
tween the disk drive and the cache memory unit, and 
further includes means of interconnecting the shared io 
memory units in the disk array control units and means 
of interconnecting the cache memory units in the disk 
array control units, thereby enabling the data read/write 
access from a channel interface unit or disk interface 
unit in one disk array control unit to a shared memory ^5 
unit or cache memory unit in other disk array control unit. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0028] 20 

Fig. 1 is a block diagram showing the arrangement 
of a disk array controller based on this invention; 
Fig. 2 is a block diagram showing the arrangement 
of a conventional disk array controller; 25 
Fig. 3 is a block diagram showing the arrangement 
of another conventional disk array controller; 
Fig. 4 is a block diagram showing the arrangement 
of still another conventional disk array controller; 
Fig. 5 is a block diagram showing the arrangement 30 
of another disk array controller based on this inven- 
tion; 

Fig. 6 Is a block diagram showing the arrangement 
of still another disk array controller based on this 
invention; 35 
Fig. 7 is a block diagram showing the detailed ar- 
rangement of the disk array control unit shown in 
Fig. 1; 

Fig. 8 is a block diagram showing the arrangement 
of a disk array controller made up of two disk array 40 
control units shown in Fig. 7 connected together; 
Fig. 9 is a block diagram showing the detailed ar- 
rangement of the disk array control unit shown in 
Fig. 5; 

Fig. 1 0 is a block diagram showing the arrangement ^5 
of a disk array controller made up of two disk array 
control units shown in Fig. 9 connected together; 
Fig. 11 is a block diagram showing the detailed ar- 
rangement of the disk array control unit shown in 
Fig. 6; 50 
Fig. 1 2 is a block diagram showing the arrangement 
a disk array controller made up of two disk array 
control units shown in Fig. 11 connected together; 
Fig. 1 3 is a block diagram showing the arrangement 
of a disk array controller made up of three disk array 55 
control units shown in Fig. 7 connected together; 
Fig. 1 4 is a perspective view showing the structure 
of a disk array control unit built on a rack based on 



this invention; 

Fig. 15 is a perspective view showing the structure 
of a disk array controller made up of two disk array 
control units which are built on racks and Intercon- 
nected; 

Fig. 1 6 is a perspective view showing the structure 
of a disk array control unit built on a different rack 
based on this invention; 

Fig. 17 Is a perspective view showing the structure 
of a disk array controller made up of four disk array 
control units which are built on one rack based on 
this invention; 

Fig. 18 is a block diagram showing the wiring 
scheme for three disk array control units based on 
inter-rack switches; 

Fig. 19 is a perspective view of the structure of the 
wiring shown in Fig. 18 based on this invention; 
Fig. 20 Is a block diagram showing the connection 
of three disk array control units based on this inven- 
tion; 

Fig. 21 is a diagram showing an example of the data 
fonnat of information of path switching which is giv- 
en to the switch box; and 

Fig. 22 is a table showing an example of the refer- 
ence table of information of path switching provided 
In the switch box. 

DESCRIPTION OF THE PREFERRED 
EMBODIMENTS 

[0029] In the following embodiments of this invention, 
hard disk drives are adopted for a large-capacity data 
storage. However, it is not confined in this invention to 
hard disk drives, but it can be DVDs for example. 
[0030] As a preferred embodiment of this invention, 
the channel interface units and disk interface units and 
the cache memory units in the multiple disk array control 
units are interconnected by a switch-based interconnec- 
tion which extends across the border of disk array con- 
trol units, and the channel interface units and disk inter- 
face units and the shared memory units are intercon- 
nected by another switch-based interconnection which 
extends across the border of disk array control units. 
[0031] As another preferred embodiment, the channel 
interface units and disk interface units and the cache 
memory units in the multiple disk array control units are 
interconnected by a switch -based interconnection 
which extends across the border of disk array control 
units, and the channel interface units and disk interface 
units and the shared memory units are connected di- 
rectly for the portion within each disk array control unit 
and interconnected for the portion between the shared 
memory units between the disk array control units by a 
switch-based interconnection which extends across the 
border of disk array control units. 
[0032] As another preferred embodiment, the channel 
interface units and disk interface units and the cache 
memory units in the multiple disk array control units are 



15 



4 



BNSDOCID: <EP 1132805A2_L> 



7 



EP1 132 805 A2 



8 



connected directly for the portion within each disk array 
control unit and interconnected for the portion between 
the cache nnemory units between the disk array control 
units by a switch-based interconnection which extends' 
across the border of disk array control units, and the 5 
channel interface units and disk interface units and the 
shared memory units are connected directly for the por- 
tion within each disk array control unit and interconnect- 
ed for the portion between shared memory units be- 
tween the disk array control units by a switch-based in- 
terconnection which extends across the border of disk 
array control units. 

[0033] With respect to the data read/write operation 
between host computers and the hard disk drives, the 
inventive disk array controller includes a plurality of disk 
array control units, each having channel interface units 
for interfacing with the computers, disk interface units 
for interfacing with the hard disk drives, a cache memory 
unit for storing temporarily data which is written to or 
read out of the hard disk drives, a shared memory unit 
for storing control infomnation of data transfer between 
the channel interface units and disk interface units and 
the cache memory unit and control Infonnatlon for the 
hard disk drives, means of connecting the channel in- 
terface units and disk interface units to the cache mem- 
ory unit, and means of connecting the channel interface 
units and disk interface units to the shared memory unit, 
and Implementing data read/write in response to a data 
read/write request from the host computer by operating 
on the channel interface units to transfer data between 
the interface with the host computer and the cache 
memory unit and operating on the disk interface units to 
transfer data between the hard disk drive and the cache 
memory unit, and further includes means of intercon- 
necting the shared memory units in the disk array control 
units and means of interconnecting the cache memory 
units in the disk array control units, thereby enabling the 
data read/write access from a host computer which is 
connected only to one disk array control unit to a hard 
disk drive which is connected only to other disk array 
control unit by way of the interconnection means. 
[0034] Other problems and schemes of solving the 
problems based on the present invention will become 
apparent from the following description of embodiments 
taken in conjunction with the accompanying drawings. 
[0035] Embodiments of the present invention will be 
explained with respect to the drawings. 

[Embodiment 1] 

[0036] Fig. 1 , Fig. 7 and Fig. 8 show an embodiment 
of this invention. Although devices of interconnection 
used in this embodiment are based on switches, these 
devices which function to transfer control information 
and data between two units may othenvise be buses for 
example. 

[0037] A disk array controller 1 of this embodiment is 
made up of multiple disk array control units 1 -1 as shown 



in Fig. 1. Each disk array control unit 1-1 includes inter- 
face units (channel IF units) 11 for interfacing with host 
computers 50, interface units (disk IF units) 12 for inter- 
facing with hard disk drives 5, shared memory units 13, 
and cache memory units 14. The channel IF units 11 
and disk IF units 12 and the shared memory units 13 
are connected by an interconnection 210 which is 
shared by the disk array control units 1 -1 , and the chan- 
nel IF units 11 and disk IF units 12 and the cache mem- 
ory units 14 are connected by another interconnection 
220 which is shared by the disk anray control units 1 -1 . 
Accordingly, all channel IF units 11 and disk IF units 12 
can access to alt shared memory units 13 or all cache 
memory units 14 via the interconnection 210 or 220. 
[0038] One disk array control unit, which may be ei- 
ther built on a rack or built as a module, can operate as 
a stand-alone disk array controller. Each disk array con- 
trol unit is assumed to be built on an individual rack in 
the explanation of Fig. 7. 

[0039] Fig. 7 shows a specific internal arrangement of 
the disk array control unit 1 -1 . 

[0040] The disk array control unit 1-1 includes two 
channel IF units 11 for interfacing with host computers 
50, two disk IF units 12 for interfacing with the hard disk 
drives 5, two shared memory path switches (SM-SWs) 
110, two cache memory path switches (CM-SWs) 111 , 
two shared memory units 13, two cache memory units 
14, shared memory (SM) access paths 135 and 136, 
cache memory (CM) access paths 137 and 138, inter- 
unit SM paths 141, and inter-unit CM paths 142. The 
inter-unit SM paths and inter-unit CM paths connect be- 
tween the disk array control units. 
[0041] The channel IF unit 11 includes two host IFs 
102 for interfacing with the host computers 50, two mi- 
croprocessors 101 which control the data transaction 
with the host computers 50, an SM access controller 1 04 
which controls the access to the shared memory units 
13, and a CM access controller 105 which controls the 
access to the cache memory units 14. It implements the 
data transfer between the host computers 50 and the 
cache memory units 1 4 and the transfer of control infor- 
mation between the microprocessors 101 and the 
shared memory units 1 3. The microprocessors 101 and 
host IFs 102 are connected by an internal bus 106. The 
CM access controller 105 is connected directly to the 
two host Ifs 102. The SM access controller 104 is con- 
nected directly to the two microprocessors 101 . 
[0042] The disk IF unit 12 includes two drive IFs 103 
for interfacing with hard disk drives 5, two microproces- 
sors 1 01 which control the data transaction with the hard 
disk drives 5, an SM access controller 104 which con- 
trols the access to the shared memory units 13, and a 
CM access controller 105 which controls the access to 
the cache memory units 14. It implements the data 
transfer between the hard disk drives 5 and the cache 
memory units 14 and the transfer of control information 
between the microprocessors 101 and the shared mem- 
ory units 13. The microprocessors 101 and drive IFs 103 
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are connected by an internal bus 106, the CM access 
controller 105 is connected directly to the drive IFs 103, 
and the SM access controller 1 04 Is connected directly 
to the nnlcroprocessors 101 . The disk IF unit 12 also im- 
plement the RAID function. 

[0043] The shared memory unit 13 includes a shared 
memory (SM) controller 1 07 and a memory module 1 09, 
and it stores control information of the disk array control 
unit 1-1 (e.g., infomnation on data transfer control be- 
tween the channel IF units 11 and disk IF units 12 and 
the cache memory units 14, and control information for 
data recorded on the hard disk drives 5). 
[0044] The cache memory unit 14 includes a cache 
memory (CM) controller 1 08 and a memory module 1 09, 
and It temporarily stores data to be recorded on the hard 
disk drives 5. 

[0045] The SM access controller 1 04 is connected to 
the two SM-SWs 1 1 0 which is different from each other 
by two SM access paths 135, and the SM-SWs 11 0 are 
connected to the two SM controllers 1 07 which is differ- 
ent from each other by two access paths 136. Accord- 
ingly, each SM controller 1 07 has the connection of two 
access paths 136 coming from the two SM-SWs 110, 
enabling each SM controller 104 to have two access 
routes to each SM controller 107. In consequence, in 
the event of a fault on one access path or in one SM- 
SW 110, the shared memory units 13 are kept accessi- 
ble through another route. 

[0046] The CM access controller 1 05 is connected to 
the two CM-SWs 111 by two CM access paths 137, and 
the CM-SWs 111 are connected to the two CM control- 
lers 1 08 by two access paths 1 38. Accordingly, each CM 
controller 108 has the connection of two access paths 
138 coming from the two CM-SWs 111, enabling each 
CM controller 105 to have two access routes to each 
CM controller 108. In consequence, in the event of a 
fault on one access path or in one CM-SW 111, the 
cache memory units 1 4 are kept accessible through an- 
other route. 

[0047] The SM-SW 110 has the connection offourSM 
access paths 135 coming from the two channel IF units 
1 1 and two disk IF units 1 2, and also has the connection 
of two access paths 1 36 going to the two shared mem- 
ory units 13. The SM-SW 110 also has the connection 
of two inter-unit SM paths 141 for the connection with 
SM-SWs 110 of other disk array control unit 1-1 . These 
inter-unit SM paths 141 may be designed differently for 
input and output, or may be designed equally for the bi- 
directional information transfer. 

[0048] The CM-SW 1 1 1 has the connection of four CM 
access paths 137 coming from the two channel IF units 
1 1 and two disk 1 F units 1 2, and also has the connection 
of two access paths 1 38 going to the two cache memory 
units 14. The CM-SW 111 also has the connection of 
two inter-unit CM paths 142 for the connection with CM- 
SWs 1 1 1 of other disk array control unit 1 -1 . These inter- 
unit CM paths 142 have the same design as the Inter- 
unit SM paths 141. 



[0049] Based on the above-mentioned access path 
connection of the SM-SWs 1 1 0 and CM-SWs 111, these 
SWs 110 and 111 have a function of directing access 
requests from the channel IF units 11 and disk IF units 

5 12 over the four access paths to the two access paths 
of the shared memory units 13 or cache memory units 
1 4 within the self disk array control unit 1 -1 and the two 
Inter-unit access paths of the shared memory units 13 
or cache memory units 1 4 of other disk an-ay control unit 

10 1-1. 

[0050] In the arrangement of Fig. 7, the SM-SWs 1 1 0 
are connection nodes between the channel IF units 11 
and disk IF units 12 and the shared memory units 13, 
and the CM-SWs 111 are connection nodes between the 
15 channel IF units 11 and disk IF units 12 and the cache 
memory units 14. 

[0051 ] Fig. 8 shows an example of the disk array con- 
troller 1 which is configured by connecting two disk array 
control units 1 -1 shown in Fig. 7. The two disk array con- 

20 trol units 1-1-1 and 1 -1 -2 have their SM-SWs 1 1 0 inter- 
connected by inter-unit SM paths 141 via inter-unit SM- 
SWs 121. Similarly, the CM-SWs 111 of the disk an-ay 
control units 1-1-1 and 1 -1 -2 are interconnected by inter- 
unit CM paths 1 42 via inter-unit CM-SWs 1 22. The SWs 

25 121 and 122 are mounted as switch boxes. 

[0052] In case, as In this embodiment, the disk array 
controller is made up of two disk array control units 1-1 , 
these units can be connected directly through the inter- 
unit paths instead of using the inter-unit SWs 121 and 

30 122 without imposing any problem on this invention, 
while eliminating the overhead process of data transfer 
in the inter-unit SWs 121 and 122. 
[0053] The disk an-ay controller 1 shown in Fig. 8 will 
be able to include three or more disk array control units 

35 1-1 by designing the inter-unit SWs 121 and 122 to have 
an increased number of path connection. The number 
of the path, which can be mounted to the inter-unit SWs 
121 and 122, is physically limited. For connecting disk 
array control units 1 -1 in excess of the limit in the number 

40 of path connection of the inter-unIt SWs 121 and 122, 
they cannot be connected by one inter-unit SW. In the 
case, the inter-units SWs are used in multiple stages to 
Increase the number of the disk array controller connect- 
ed. 

45 [0054] Fig. 1 9 shows an example of a disk array con- 
troller which is made up of three disk array control units 
built on individual racks and interconnected through a 
switch box, as will be explained in detail later 
[0055] For connecting three or more disk array control 

50 units 1-1, the SM-SWs 110 and CM-SWs 111 in all units 
1-1 are each connected in a loop fashion as shown in 
Fig. 13. This interconnecting scheme enables the loop 
connection of multiple disk array control units 1-1 with- 
out using the inter-unit SWs 121 and 122. Specifically, 

55 the inter-unit SM paths of adjacent units 1-1 and the in- 
ter-unit CM paths of adjacent units 1-1 are each con- 
nected by connectors. Preferably, these connectors of 
the inter-unit SM paths 141 and inter-unit CM paths are 
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fixed on the exterior wall of the disk array control units 
1-1 for the convenience of the additional installation of 
units 1-1 , although it is not shown in the figure. 
[0056] In the arrangement of Fig. 8, the SM-SWs 1 1 0, 
inter-unit SM paths 141 and inter-unit SM-SWs 121 in 
unison constitute the interconnection 210 shown in Fig. 
1, and the CM-SWs 111, inter-unit CM paths 142 and 
inter-unit CM-SWs 1 22 In unison constitute the intercon- 
nection 220. 

[0057] An example of the operation of the arrange- 
ment shown in Fig. 8 for reading out data from a host 
computer 50 which is connected to the disk array control 
unit 1-1-1 will be explained. 

[0058] Initially, the host computer 50 issues a data 
read-out request to the channel IF unit 11 in the disk 
array control unit 1 -1 -1 to which it is connected. The mi- 
croprocessors 101 in the channel IF unit 11 which has 
received the request makes access to both shared 
memory units 13 in both disk array control units 1-1-1 
and 1-1-2 thereby to find as to which hard disk drive 5 
stores the requested data. The search is executed 
based on the conversion table stored in the shared 
memory units 1 3 indicative of the correspondence of the 
address of the requested data and hard disk drives 
which stores the data. 

[0059] Subsequently, the microprocessors 101 in the 
channel IF unit 11 which has received the request 
makes access to both shared memory units 13 in both 
disk array control units 1-1-1 and 1-1-2 thereby to find 
as to whether or not the requested data is present in the 
cache memory units 14 in the disk array control units 
1-1-1 and 1-1-2. The search is based on information 
stored in the shared memory units 13 indicative of the 
directory of data of the cache memory units 14. 
[0060] In one case of finding the requested data in the 
cache memory unit 14 of the disk array control unit 
1-1-1, the data is transferred to the channel IF unit 11 
via the CM-SW 111 within the unit 1-1-1, and then it is 
sent to the host computer 50. In other case of finding 
the data in the cache memory unit 14 of the disk array 
control unit 1 -1 -2, the data is transferred to the channel 
IF unit 11 by way of the CM-SW 111 within the unit 1-1-2, 
the inter-unit CM-SW 1 22 and the self CM-SW 111, and 
then it is sent to the host computer 50. 
[0061] In case the request data does not exist in any 
cache memory unit 14, the microprocessor 101 issues 
a command to another microprocessor 101 in the disk 
IF units 12 which Is connected with the hard disk drive 
5 where the requested data Is stored, thereby operating 
it to read out and store the data from the hard disk drive 
5 into one cache memory unit 1 4. On receiving the com- 
mand, the microprocessor 101 in the disk IF unit 12 
reads the requested data out of the hard disk drive 5 
and stores the data in the cache memory unit 14 of the 
disk array control unit 1-1-1 or 1 -1 -2. 
[0062] In one case of storing the data in the cache 
memory unit 1 4 in the disk array control unit 1 -1 -2 which 
is connected to the hard disk drive 5 where the request- 



ed data is stored, the data is transferred to the cache 
memory unit 14 via the CM-SW 111 in the disk array 
control unit 1 -1 -2. In another case of storing the request- 
ed data in the cache memory unit 14 in another disk ar- 

5 ray control unit 1-1-1 which is different from the disk ar- 
ray control unit 1 -1 -2 connected with the hard disk drives 
5 in which the requested data is stored, the data is trans- 
ferred by way of the CM-SWs 111 of both units 1-1-2 
and 1-1-1 and the inter-unit CM-SW 122. 

10 [0063] After the requested data has been stored in the 
cache memory unit 14, the microprocessor 101 in the. 
disk IF unit 1 2 Indicatesthe cache memory unit 14 where 
the data has been stored to the command-Issuing mi- 
croprocessor 1 01 in the channel IF unit 11. In response 

^5 to this Indication, the microprocessor 1 01 in the channel 
IF unit 1 1 read the data out of the cache mempry unit 1 4 
where the data has been stored and sends it to the host 
computer 50. 

[0064] According to this embodiment, a host compu- 
20 ter 50 can transact data with a hard disk drive 5 by simply 
issuing an access request to the disk array control unit 
1-1 to which it is connected without being concerned 
with as to which hard disk drive 5 of which disk array 
control unit 1-1 is stored the data in need. Namely, the 
25 host computer 50 can deal with the multiple disk array 
control units 1-1 virtually as a single disk array controller 
1. 

[0065] Furthermore, in the case of reading out data 
from a hard disk drive 5 connected to a disk array control 

30 unit 1 -1 which is different from the unit 1 -1 that has re- 
ceived the request, the data can be read out by way of 
the internal interconnection and cache memory unit 14, 
instead of moving the data through the channel IF units 
11 of both disk array control units 1-1 , whereby the de- 

35 terio ration of data read/write perfomnance can be alle- 
viated. 

[Embodiment 2] 

40 [0066] Fig. 5, Fig. 9 and Fig. 1 0 show another embod- 
iment of this Invention. 

[0067] In Fig. 5, a disk array controller 1 which is made 
up of multiple disk array control units 1-2 is arranged 
identicallyto the preceding Embodiment 1 shown in Fig. 

"^5 1 except for the connection between the channel IF units 
1 1 and disk IF units 1 2 and the shared memory units 1 3. 
In each disk array control unit 1-2, the channel IF units 
1 1 and disk IF units 1 2 and the shared memory units 1 3 
are connected directly. The shared memory units 1 3 are 

50 interconnected via an interconnection 24 between the 
disk array control units 1-2. Connecting the channel IF 
units 11 and disk IF units 12 and the shared memory 
units 13 directly in each disk array control unit 1-2 re- 
duces the access time to the shared memory units 13 

55 as compared with the connection via the interconnection 
21 0 which is the case of Embodiment 1 . 
[0068] Fig. 9 shows a specific internal arrangement of 
the disk array control unit 1 -2. The internal arrangement 
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of the disk array control unit 1-2 is identical to the pre- 
ceding Embodiment 1 shown in Fig. 7 except for the con- 
nection between the channel IF units 11 and disk IF units 

12 and the shared memory units 13. 

[0069] The disk array control unit 1-2 includes two 
channel IF units 11 for interfacing with computers 50, 
two disk IF units 12 for interfacing with hard disk drives 
5, two cache memory path switches (CM-SWs) 111, two 
shared memory units 13, two cache memory units 14, 
shared memory (SM) access paths 139, a cache mem- 
ory (CM) access paths 137 and 1 38, inter-unIt SM paths 
143, and inter-unIt CM paths 142. 
[0070] The SM access controller 1 04 is connected to 
two SM controllers 1 07 which is different from each oth- 
er by two SM access paths 139. Accordingly, the SM 
controller 107 has the connection of four SM access 
paths 139 coming from the two channel IF units 11 and 
two disk IF units 12. The SM controller 1 07 also has the 
connection of two inter-unit SM paths 143 for the con- 
nection with SM controllers 1 07 of other disk array con- 
trol unit 1-2. 

[0071] Based on the above-mentioned access path 
connection of the SM controller 107, It has a function of 
directing access requests from the channel IF units 11 
and disk IF units 12 over the four access paths to the 
access paths of the memory modules 109 and the two 
Inter-unit access paths 143 of the shared memory units 

13 In other disk an-ay control unit 1-2. 

[0072] Fig. 10 shows an example of the disk array 
controller 1 which is configured by connecting two disk 
array control units 1 -2 shown in Fig. 9. The two disk ar- 
ray control units 1-2 have their shared memory units 13 
Interconnected by inter-unit SM paths 143 via inter-unit 
SM-SWs 121. The SM controllers 1 07 work for the con- 
nection nodes of the channel IF units and disk IF units 
and the shared memory units. The remaining portion is 
Identical to the arrangement of Embodiment 1 shown in 
Fig. 8. 

[0073] In case the disk array controller Is made up of 
two disk array control units 1 -2, these units can be con- 
nected directly through the inter-unit paths Instead of us- 
ing the Inter-unit SWs 121 and 122 without imposing any 
problem on this Invention, while eliminating the over- 
head process of data transfer In the inter-unit SWs 1 21 
and 122 to improve the perfomiance, as in the case of 
Embodiment 1 . 

[0074] The disk array controller 1 shown in Fig. 1 0 will 
be able to include three or more disk array control units 
1 -2 by designing the Inter-unit SWs 1 21 and 1 22 to have 
an increased number of path connection, as in the pre- 
ceding Embodiment 1 . The number of the path, which 
can be mounted to the inter-unit SWs 121 and 122, Is 
physically limited. For connecting disk array control 
units 1 -2 in excess of the limit in the number of path con- 
nection of the inter-unit SWs 121 and 122, they cannot 
be connected by one inter-unit SW. In the case, the inter- 
units SWs are used in multiple stages. These inter-unit 
SWs are built in a switch box, as in the preceding Em- 



bodiment 1 . 

[0075] Three or more disk array control units 1 -2 can 
be chained In a loop fashion similarto the preceding Em- 
bodiment 1 shown in Fig. 13. The plural disk array con- 
5 trol units 1 -2 can be connected without using the inter- 
unlt SWs 121 and 122. 

[0076] The operation of the functional units in the disk 
array control unit 1-2 for writing or reading out data to/ 
from the hard disk drives 5 by the host computers 50 is 

10 identical to the preceding Embodiment 1 except that the 
access to the shared memory unit 13 in other disk an-ay 
control unit 1 -2 takes place via the shared memory unit 
13 in the self unit 1-2 and the Interconnection 24. 
[0077] The inter-unit SM paths 143, inter-unit SM- 

15 SWs 1 21 , and SM controller 1 07 in unison constitute the 
interconnection 24. 

[0078] According to this embodiment, a host compu- 
ter 50 can transact data with a hard disk drive 5 by simply 
issuing an access request to the disk array control unit 

20 1-2 to which it is connected without being concerned 
with as to which hard disk drive 5 of which disk an-ay 
control unit 1 -2 Is stored the data in need. Namely, the 
host computer 50 can deal with the multiple disk array 
control units 1 -2 virtually as a single disk array controller 

25 1. 

[0079] Furthemiore, In the case of reading out data 
from a hard disk drive 5 connected to a disk array control 
unit 1 -2 which Is different from the unit 1 -2 that has re- 
ceived the request, the data can be read out by way of 
30 the internal interconnection and cache memory unit 14, 
instead of moving the data through the channel IF units 
11 of both disk an-ay control units 1-2, whereby the de- 
terioration of data read/write performance can be alle- 
viated. 

35 

[Embodiment 3] 

[0080] Fig. 6, Fig. 11 and Fig. 12 show another em- 
bodiment of this invention. 

40 [0081 ] In Fig. 6, a disk array controller 1 which Is made 
up of multiple disk array control units 1-3 Is arranged 
identically to the preceding Embodiment 2 shown in Fig. 
5 except for the connectio n between th e ch an n ell F u nits 
11 and disk IF units 12 and the cache memory units 14. 

45 In each disk array control unit 1 -3, the channel IF units 
11 and disk IF units 12 and the cache memory units 14 
are connected by an interconnection 22. The cache 
memory units 14 are interconnected between disk array 
control units 1-3 by an interconnection 25. The channel 

50 IF units 11 and disk IF units 12 and the shared memory 
units 13 are connected directly, whereas the channel IF 
units 1 1 and disk I F units 1 2 and the cache memory units 
14 are connected by the interconnection 22, as in the 
case of the preceding Embodiment 2, by the following 

55 reason. The shared memory unit 13 deals with control 
data having a size of several bytes for example, where- 
as the cache memory unit 14 deals with data having a 
size of several kilobytes for example. Therefore, It is in- 
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tended to raise the throughput based on the connection 
through the interconnection 22 with a limited number of 
pins. 

[0082] By the separate provision of the interconnec- 
tion 22 which connects the channel IF units 11 and disk 
IF units 12 and the cache memory units 14 and the in- 
terconnection 25 which connects the cache memory 
units 14 between the disk array control units 1-3, data 
transfer between cache memory units 14 of different 
disk array control units 1 -3 does not preclude the host 
computers 50 from making access to the cache memory 
units 14. 

[0083] The disk IF unit implements the data transfer 

between cache memories. The function of data transfer 
between cache memory units 14 of different disk array 
control units 1 -3 is necessary for moving data to a disk 
array control unit of less-frequent access in dealing with 
concentrated accesses from host computers 50 to one 
disk array control unit 1-3. 

[0084] Fig. 11 shows a specific Internal arrangement 
of the disk array control unit 1-3. The intemal arrange- 
ment of the disk array control unit 1 -3 is identical to the 
preceding Embodiment 2 shown in Fig. 9 except for the 
connection between the channel IF units 11 and disk IF 
units 12 and the cache memory units 14. The CM con- 
trollers 1 08 work for the connection nodes of the channel 
IF units, disk IF units and cache memory units. 
[0085] The disk array control unit 1-3 includes two 
channel IF units 11 for interfacing with host computers 
50, two disk IF units 12 for interfacing with hard disk 
drives 5, two cache memory path switches (CM-SWs) 
111, two shared memory units 13, two cache memory 
units 14, shared memory (SIVI) access paths 139, cache 
memory (CM) access paths 137 and 138, inter-unit SM 
paths 143, and inter-unit CM paths 144. 
[0086] The CM access controller 1 05 is connected to 
two CM-SWs 111 by two CM access paths 137, and the 
CM-SW 111 is connected to two CM controllers 1 08 by 
two access paths 138. Accordingly, the CM controller 
1 08 has the connection of two access paths 1 38 coming 
from the two CM-SWs 111 , and it also has the connec- 
tion of two inter-unit CM paths 144 for the connection 
with CM controllers 108 of other disk array control unit 
1-3. 

[0087] Based on the above-mentioned access path 
connection of the CM controller 108, it has a function of 
directing access requests from the CM-SWs 111 over 
the two CM access paths 1 38 to the access paths of the 
memory modules 1 09 and the two inter-unit CM access 
paths 144 of the cache memory units 14 in other disk 
array control unit 1-3. 

[0088] The CM-SW 1 1 1 has the connection of four CM 
access paths 137 coming from the two channel IF units 
1 1 and two disk I F units 1 2, and also has the connection 
of two access paths 1 38 going to the two cache memory 
units 14. 

[0089] Based on this access path connection of the 
CM-SW 111, it has a function of directing access re- 



quests from the channel IF units 1 1 and disk IF units 12 
over the four CM access paths 1 37 to the two CM access 
paths 138 of the cache memory units 14. 
[0090] Fig. 12 shows an example of the disk array 

5 controller 1 which is configured by connecting two disk 
array control units 1-3 shown in Fig. 11. The two disk 
array control units 1 -3 have their cache memory units 
14 interconnected bylnter-unit CM paths 144 via inter- 
unit CM-SWs 122. The remaining portion is identical to 

10 the arrangement of Embodiment 2 shown in Fig. 10. 
[0091 ] In case the disk array controller is made up of 
two disk array control units 1 -3, these units can be con- 
nected directly through the inter-unit paths instead of us- 
ing the inter-unit SWs 121 and 122 without imposing any 

15 problem on this invention, while eliminating the over- 
head process of data transfer in the inter-unit SWs 121 
and 122 to improve the performance, as in the case of 
Embodiment 2. 

[0092] The disk array controller 1 shown in Fig. 12 will 

20 be able to include three or more disk array control units 
1-3 by designing the inter-unit SWs to have an increased 
number of path connection, as in the preceding Embod- 
iment 2. The number of the path, which can be mounted 
to the inter-unit SWs 121 and 122, is physically limited. 

25 For connecting disk array control units 1 -3 in excess of 
the limit In the number of path connection of the inter- 
unlt SWs 1 21 and 1 22, they cannot be connected by one 
inter-unit SW. In the case, the inter-units SWs are used 
in multiple stages. 

30 [0093] Three or more disk array control units 1 -3 can 
be chained in a loop fashion similarto the preceding Em- 
bodiment 1 shown In Fig. 13. The plural disk array con- 
trol units 1-3 can be connected without using the inter- 
unit SWs 121 and 122. 

35 [0094] The operation of the functional units in the disk 
array control unit 1-3 for transacting data with the hard 
disk drives 5 by the host computers 50 is identical to the 
preceding Embodiment 2 except that the access to the 
cache memory unit 1 4 in other disk array control unit 1 -3 

^0 takes place via the cache memory unit 14 in the self unit 

I -3 and the interconnection 25. 

[0095] According to this embodiment, a host compu- 
ter 50 can transact data with a hard disk drive 5 by simply 
issuing an access request to the disk array control unit 

45 1-3 to which it is connected without being concerned 
with as to which hard disk drive 5 of which disk array 
control unit 1 -3 is stored the data in need. Namely, the 
host computer 50 can deal with the multiple disk array 
control units 1 -3 virtually as a single disk array controller 

50 1. 

[0096] Furthermore, in the case of reading out data 
from a hard disk drive 5 connected to a disk array control 
unit 1-3 which is different from the unit 1 -3 that has re- 
ceived the request, the data can be read out by way of 
55 the internal Interconnection and cache memory unit 14, 
instead of moving the data through the channel IF units 

II of both disk array control units 1-3, whereby the de- 
terioration of data read/write performance can be alle- 
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viated. 

[0097] Next, examples of the usage of the embodi- 
ments of this invention will be explained. 
[0098] High-end disk an^ay controllers have the fol- 
lowing function. In the function, a data set (correspond- s 
ing to a logical volume) for one work is duplicated and 
stored, and both the original and duplicate data sets are 
updated simultaneously in the ordinary work. At the 
emergence of the need of backup for the data set for 
example, updating of the duplicate data set is suspend- io 
ed and it is used for the backup, while the original data 
set Is used continuously for the work, and the original 
and duplicate data sets are rendered the matching proc- 
ess on completion of backup. 

[0099] In a disk array controller 1, which have been is 
explained as Embodiment 1 , a method of accomplishing 
this function for storing a duplicate data set in the differ- 
ent disk array control units 1 -1 will be explained with ref- 
erence to Fig. 8. It is assumed that an original data set 
is stored on the hard disk drive 5 connected to the disk 20 
array control unit 1-1-1 and a duplicate data set is stored 
on the hard disk drive 5 connected to the disk array con- 
trol unit 1-1-2 in Fig. 8. It is also assumed that the host 
computer 50 connected to the disk array control unit 

I -1 -1 works for the ordinary work and the host computer 25 
50 connected to the disk array control unit 1 -1 -2 works 

to backup data on a tape storage unit (not shown) con- 
nected to It. 

[0100] In the ordinary work, In the event of a request 
of the host computer 50, which is connected to the disk 30 
array control unit 1 -1 -1 , to write data to an intended data 
set, the microprocessor 101 in the channel IF unit 11, 
which Is connected to the host computer 50 connected 
to the disk array control unit 1 -1 -1 , transfers and writes 
the data which is sent from the host computer 50 into 35 
the cache memory unit 14 of the disk an-ay control unit 
1-1-1. Subsequently, the microprocessor 101 issues a 
command by way of the shared memory unit 13 of the 
disk array control unit 1-1-1 to the microprocessor 1 01 
in the disk IF unit 1 2 which is connected to the hard disk 40 
drive 5 where the original data set is stored, thereby op- 
erating on it to read out data from the cache memory 
unit 14 of the disk array control unit 1-1-1, transfers the 
data to the disk IFunit 12which Is connected to the hard 
disk drive 5 where the original data set is stored, and 45 
operates on the disk IF unit 12 to transfer and write the 
data onto the hard disk drive 5. 

[0101] The microprocessor 101 in the channel IF unit 

II of the disk array control unit 1-1-1, which monitors 
the updating of data of the original data set, stores in 50 
the shared memory units 13 of the disk array control unit 
1-1-1 the control information indicative of the quantity of 
updated data of the original data set. When the quantity 

of updated data exceeds a predetermined value, the mi- 
croprocessor 101 Issues a command to the microproc- 55 
essor 101 in the disk IF unit 12 which is connected with 
the hard disk drive 5 where the original data set is stored, 
thereby operating on it to revise the duplicate data set 



to match with the updated content of the original data 
set. 

[0102] The command receiving microprocessor 101 
reads out the updated data from the hard disk drive 5 
and converts the addresses of updated data into ad- 
dresses of the duplicate data set. It transfers and writes 
the updated data via the CM-SW 111 of the disk array 
control unit 1-1-1, inter-unit CM-SW 122 and CM-SW 
111 of the disk array control unit 1-1-2 into the cache 
memory unit 14 of the disk array control unit 1-1-2. 
[0103] Subsequently, the microprocessor 101 reads 
out the updated data from the cache memory unit 14, 
transfers the data to the disk IF unit 1 2 which Is connect- 
ed to the hard disk drive 5 where the duplicate data set 
Is stored, and the data is transferred and stored onto the 
hard disk drive 5. 

[0104] Based on the foregoing operation, the disk ar- 
ray controller maintains the original data set and dupli- 
cate data set during the ordinary work. 
[0105] In the event of a request of the host computer 
50, which is connected to the disk array control unit 
1-1-2, to backup data for an intended data set, the mi- 
croprocessor 101 in the channel IFunit 11 connected to 
the host computer 50 issues a command by way of the 
shared memory unit 13 of the disk array control unit 
1-1-1 to the microprocessor 101 In the channel IF unit 
11 which is connected to the host computer 50 having 
the ordinary work, thereby operating on it to halt the up- 
dating of the duplicate data set. The command receiving 
microprocessor 1 01 halts the updating of duplicate data 
set. The microprocessor 101 in the channel IF unit 11 , 
which is connected to the backup requesting host com- 
puter 50, issue a command by way of the shared mem- 
ory unit 13 of the disk array control unit 1-1-2 to the mi- 
croprocessor 101 in the disk IF unit 12 which is connect- 
ed to the hard disk drive 5 where the duplicate data set 
is stored, thereby operating on It to read out and transfer 
the duplicate data set from the hard disk drive 5 to the 
disk IF unit 12, from which it is transferred and written 
into the cache memory unit 14 of the disk array control 
unit 1-1-2. 

[0106] Subsequently, the microprocessor 101 in the 
channel IF unit 1 1 reads out the duplicate data set from 
the cache memory unit 14 of the disk array control unit 
1-1-2 and transfers it to the channel IF unit 11, from 
which it Is sent to the backup requesting host computer 
50. 

[0107] On completion of backup of data set, the mi- 
croprocessor 1 01 in the channel 1 F unit 1 1 , which is con- 
nected to the backup requesting host computer 50, is- 
sues a command to the microprocessor 1 01 in the chan- 
nel IF unit 11 which is connected to the host computer 
50 having the ordinary wori< through the shared memory 
units 13 of the disk an-ay control unit 1 -1 -1 , thereby op- 
erating on it to revise the duplicate data set to match 
with the content of the original data set which has been 
updated during the backup process. This operation Is 
identical to the data updating in the ordinary work ex- 
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plained previously. 

[0108] In accomplishing the above-nr»entioned func- 
tion, this operational scheme enables the nnovement of 
data between the two disk array control units 1 -1 -1 and 
1-1-2 via the internal interconnections and cache mem- 
ory units 14, eliminating the need of data transfer via the 
channel IF units of both disk array control units 1-1-1 
and 1 -1 -2, whereby the deterioration of system perform- 
ance during the implementation of this function is alle- 
viated and thus the deterioration of efficiency of the us- 
er's ordinary work is prevented. 
[0109] This operational scheme can be carried out 
equally In the disk array controller 1 of Embodiment 2 
and Embodiment 3 without problems, and the same ef- 
fectiveness is attained. 
[01 1 0] Other usages Include the following. 
[0111] In the disk array controllers 1 of Embodiment 
1 , Embodiment 2 and Embodiment 3, when a host com- 
puter 50 sends a data read request, for example, to a 
disk array control unit and if the data is absent on the 
hard disk drive 5 connected to the unit, it is necessary 
to read out the data via an interconnection from another 
disk an-ay control unit connected with a hard disk drive 
5 where the data is stored and send the data to the re- 
questing host computer 50. The data read/write opera- 
tion across the border of disk array control units, as in 
this case, results in the deterioration of performance as 
compared with the operation within a unit. 
[0112] For avoiding the data transfer between units, 
it is necessary for the system to have a function of mov- 
ing data, which is accessed frequently by one host com- 
puter 50 and is stored on a hard disk drive connected to 
a disk array control unit other than the control unit con- 
nected to the one host computer 50, to the hard disk 
drive 5 of the control unit connected to the one host com- 
puter 50. 

[01 13] This data moving function implemented by the 
disk an-ay controller 1 of Embodiment 1 will be explained 

with reference to Fig. 8. 

[0114] The microprocessor 101 in the channel IF unit 
11 monitors the frequency of access to data sets (cor- 
responding to logk:al volumes) on all hard disk drives 5, 
and stores control Information indicative of the access 
frequency of the data sets in the shared memory units 
13 within the same disk array control unit 1-1-1 . 
[01 15] If the host computer 50 connected to the disk 
array control unit 1-1-1 makes access to the data set on 
the hard disk drive 5 connected to another disk array 
control unit 1 -1-2 to concentrate the access on the data 
set, the access frequency is exceed a predetermined 
rate, the microprocessor 101 in the channel IF unit 11 
of the disk array control unit 1-1-1 issues a command 
through the shared memory unit 1 3 in the other disk ar- 
ray control unit 1-1-2 to the microprocessor 101 in the 
disk IF unit 12 which is connected with the hard disk 
drive 5 where the data set is stored, thereby operating 
on it to read out the data set and transfer and write the 
data set into the cache memory unit 1 4 of the disk array 



control unit 1-1-2. 

[0116] Subsequently, the microprocessor 101 in the 
channel IF unit 11 of the disk array control unit 1-1-1 
reads out and transfers the relevant data from the cache 
5 memory unit 14 of the disk array control unit 1-1 -2 to the 
cache memory unit 14 of the disk array control unit 
1-1-1. The microprocessor 101 subsequently issues a 
command by way of the shared memory unit 13 in the 
disk array control unit 1-1-1 to the microprocessor 101 
10 in the disk IF unit 12 of the disk array control unit 1 -1-1 , 
thereby operating on it to read out, transfer and write the 
relevant data from the cache memory unit 14 of the disk 
array control unit 1-1-1 onto the hard disk drive 5. 
[0117] This operational scheme enables the move- 
rs rnent of data between two disk array control units 1-1 
via the internal interconnections and cache memory 
units 14, eliminating the need of data transfer via the 
channel IF units of both diskarray control units, whereby 
the deterioration of system performance during the data 
20 movement Is alleviated and thus the deterioration of ef- 
ficiency of the user's ordinary work is prevented. 
[0118] This operational scheme can be carried out 
equally in the disk an^ay controller 1 of Embodiment 2 
and Embodiment 3 without problems, and the same ef- 
25 fectiveness is attained. 

[01 1 9] Next, the structure of the disk array controllers 
of Embodiment 1 , Embodiment 2 and Embodiments will 
be explained. 

[0120] Fig. 14 shows an example of the structure of 
30 the disk array control unit 1-1 of Embodiment 1 shown 
in Fig. 7 built on a rack 201 . The channel IF units 11 are 
arranged in channel IF packages (PKs) 311 , the disk IF 
units 12 are arranged in disk IF packages 312, the SM- 
SWs 110 and CM-SWs 111 are arranged in switch pack- 
35 ages 320, and the shared memory units 13 and cache 
memory units 14 are arranged in memory packages 
330. These packages are plugged into the sockets on 
a back board 340, which has the wiring of the SM access 
paths 1 35 and 1 36 and the CM access paths 1 37 and 
40 138. 

[0121] The switch packages 320 has the connection 
of cables of the inter-unit SM paths 1 41 and cables of 
the inter-unit CM paths 142, and these cables have oth- 
er ends connected to connectors 221 and 222, respec- 
45 tively, on the side wall of the rack 201 . The cables are 
not shown. A power unit 350 supplies electric power to 
the packages. The disk array control unitbuittona rack 
as shown operates as a stand-alone disk array control- 
ler. 

50 [0122] Fig, 15 shows an example of the interconnec- 
tion of two sets of the disk array control unit built on the 
rack 201 shown in Fig. 14. 

[0123] A switch box 210 houses the inter-unit SM- 
SWs 121 and inter-unit CM-SWs 122 shown In Fig. 8. 
55 The inter-unit SM paths 141 coming from the Inter-unit 
SM-SWs 121 are connected to connectors 221 , and the 
inter-unit CM paths 142 coming from the inter-unit CM- 
SWs 122 are connected to connectors 222. 
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[0124] For interconnecting the two disk array control 
units on the racks 201 and 202, the inter-unit SM path 
connectors 221 on the rack 201 are connected to the 
connectors 221 on the switch box 21 0 with cables 231 , 
and the inter-unit CM path connectors 222 on the rack s 
201 are connected to the connectors 222 on the switch 
box 21 0 with cables 232. Similarly, the connectors 221 
on the rack 202 are connected to the connectors 221 on 
the switch box 210 with cables 231, and the inter-unit 
CM path connectors 222 on the rack 202 are connected io 
to the connectors 222 on the switch box 21 0 with cables 
232. 

[0125] The disk array controller built on two racks in 
this manner can deal with an increased number of host 
computer channels and an increased storage capability. ^5 
[0126] By designating the racks 201 and 202 to be a 
basic and extension racks, respectively, and accommo- 
dating the switch box 21 0 inside the extension rack 202, 
it becomes possible to eliminate the floor space for the 
switch box 210 without increasing the manufacturing 20 
cost of the basic rack 201 . 

[01 27] This structural scheme can also be applied to 
the disk array controllers of Embodiment 2 and Embod- 
iment 3 without any problem. 

[0128] Fig. 1 8 shows the interconnection of three disk 25 
array control units by inter-unIt SM-SWs 121 and CM- 
SWs 122. Each switch (SW) which connects three disk 
array control units must have a larger capability as com- 
pared with the case of connecting two units. Specifically, 
disk array control units 1-1-1,1-1-2 and 1-1-3 are inter- 30 
connected through inter-unit SM paths 141 and inter- 
unit CM paths 142 via the inter-unit SM-SWs 121 and 
CM-SWs 122, and function as a disk array controller. 
[0129] Fig. 19 shows the structure of this disk array 
controller. The switch box 210 has a separate rack. The 35 
disk an^ay control units built on individual racks 201 ,202 
and 203 are interconnected by inter-unit SM path cables 

231 and connectors 221 and inter-unit CM path cables 

232 and connectors 222. Providing more capability and 
more connectors for the switch box 21 0 for connecting 40 
four or more disk an^ay control units facilitates the addi- 
tional installation in the future. 

[0130] Fig.21 shows the fomri at of data which is trans- 
ferred through the switch box 210. Data has a form of 
packet, and it consists of a destination address field 401 , 45 
command field 402 and data field 403. The address is 
of the shared memory and cache memory. 
[0131] Fig. 22 shows a reference table 410 for path 
switching provided in the switch box 21 0, This table con- 
tains the correspondence between the destination ad- so 
dress and the port number of the disk array control unit. 
The switch box 21 0 makes reference to the path switch- 
ing table to determine a destination unit based on the 
address 401 in the packet data 400 and implements the 
path switching. 55 
[0132] Additional installation of disk array control 
units is carried out based on the following procedure. If 
the switch box 210 has spare connectors for additional 



disk array control units, cables 231 and 232 are con- 
nected to these connectors. If there are no spare con- 
nectors, another switch box is added in a multi-stage 
configuration and cables 231 and 232 are connected to 
it. The path switching table 41 0 in the switch box 21 0 is 
revised to include the addresses and port numbers for 
the additional disk array control units. An alternative de- 
sign of the table is to preset additional addresses with 
flags so that address flags are turned on for the addi- 
tional unit installation. 

[0133] Fig. 20 shows another unit interconnection 
scheme. In a disk an^ay controller, three disk array con- 
trol units are interconnected in series. SM-SWs 1 10 and 
CM-SWs 111 have a bridge function of transferring in- 
coming data intact to other disk array control unit. These 
SM-SWs 110 and CM-SWs 111 may be replaced with 
common buses, by which multiple disk array control 
units are interconnected. 

[0134] Furthertheotherembodlmentwillbeshownas 
follows. 

[0135] As shown in Fig. 16, a disk array control unit 
made up of a minimal number of packages (PKs), which 
are used in the unit rack 201 shown in Fig. 14, is built 
on a rack 205. 

[01 36] As shown in Fig. 1 7, two or more unit racks 205 
and a switch box 210 are built on a large rack 206, and 
the units on the racks 205 are interconnected via the 
switch box 21 0 based on the interconnection scheme of 
the sixth embodiment. This structure is suitable for mid- 
dle-scale to large-scale disk array controllers. The rack 
205 has the same role for the disk array control unit as 
of the foregoing one, or alternatively the disk array con- 
trol unit may have a module stnjcture. Each disk array 
control unit on the rack 206 may include an individual 
power unit, or all units may be supplied with powerfrom 
a single power unit. 

[0137] According to the present invention, it becomes 
possible to construct a disk array system which is based 
on the operation of multiple disk array control units as a 
single disk array controller and is capable of alleviating 
the deterioration of perfomiance caused by the move- 
ment of data between the disk array control units. 



Claims 

1 . A disk array controller (1 ) comprising: 

a plurality of disk array control units (1-1 , 1-2, 
1-3), each having one or more channel inter- 
face units (11 ) for interfacing with a host com- 
puter (50); one or more disk interface units (1 2) 
for interfacing with disk drives (5); a cache 
memory unit (14) for temporarily storing data 
which is written to or read out of said disk drives 
(5); and a shared memory unit (13) for storing 
control information of data transfer between 
said channel interface unit (11) and disk inter- 
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4. 



face unit (12) and said cache memory unit (14) 
and control infomriatlon for said disk drives (5), 
said disk array control unit (1-1 , 1-2, 1-3) imple- 
menting the data read/write operation in re- 
sponse to a data read/write request from said 
host computer (50) by operating on said chan- 
nel Interface unit (11) to transfer data between 
said interface (11) with said host computer (50) 
and said cache memory unit (1 4) and operating 
on said disk interface unit (12).to transfer data 
between said disk drive (5) and said cache 
memory unit (14), 

said channel interface units (11) and disk inter- 
face units (12) and said cache memory units 
(14) in said disk array control units (1-1, 1-2, 
1-3) having their connection nodes intercon- 
nected through a first interconnection (25, 220) 
across the border of disk array control units 
(1-1, 1-2, 1-3), 

said channel interface units (11) and disk inter- 
face units (12) and said shared memory units 
(13) in said disk array control units (1-1, 1-2, 
1-3) having their connection nodes intercon- 
nected through a second interconnection (24, 
210) across the border of disk array control 
units. 

A disk array controller according to claim 1 , wherein 
said channel interface units (11) and disk interface 
units (12) and said cache memory units (14) in said 
multiple disk array control units (1-1, 1-2, 1-3) are 
connected through said first interconnection (220), 
and said channel interface units (11) and disk inter- 
face units (12) and said shared memory units (13) 
are connected through said second interconnection 
(210). 

A disk array controller according to claim 1 , wherein 
said channel interface units (11) and disk interface 
units (12) and said cache memory units (14) In said 
multiple disk array control units (1-1, 1-2, 1-3) are 
connected by said first interconnection (220), and 
said channel interface units (11) and disk interface 
units (12) and said shared memory units (13) are 
connected directly for the portion within each disk 
array control unit. 

A disk array controller according to claim 1 , wherein 
said channel interface units (11) and disk interface 
units (12) and said cache memory units (14) In said 
multiple disk array control units (1-1, 1-2, 1-3) are 
connected for the portion within each disk array 
controt unit through a third interconnection in each 
disk array control unit, and said channel interface 
units (11) and disk interface units (12) and said 
shared memory units (1 3) are connected directly for 
the portion within each disk array control unit. 



5. A disk array control unit comprising: 

one or more channel interface units (1 1 ) for in- 
terfacing with a host computer (60); one or 

5 more disk interface units (12) for interfacing 

with disk drives (5); a cache memory unit (14) 
for temporarily storing data which is written to 
or read out of said disk drives (5); and a shared 
memory unit (13) for storing control information 

10 of data transfer between said channel interface 

unit (11) and disk interfaice unit (12) and said 
cache memory unit (14) and control information 
for said disk drives (5), 

said disk array control unit implementing the 
IS data read/write operation in response to a data 

read/write request from said host computer (50) 
by operating on said channel Interface unit (11) 
to transfer data between said interface with said 
host computer (50) and said cache memory unit 
20 (14) and operating on said disk interface unit 

(1 2) to transfer data between said disk drive (5) 
and said cache memory unit (14), 

said disk array control unit having, on the con- 
nection node of said channel interface unit (11) 

25 and disk interface unit (12) and said cache 

memory unit (14), a first connection path (25, 
220) for the connection with other disk array 
control unit, and having, on the connection 
node of said channel interface unit (11) and disk 

30 interface unit (1 2) and said shared memory unit 

(1 3) , a second connection path (24, 21 0) for the 
connection with the other disk array control 
unit. 

35 6. A disk array control unit according to claim 5, where- 
in said disk array control unit is built on a rack (205), 
and said first and second connection paths have 
connectors (221 , 222). . 

40 7. A disk array controller comprising: 

a plurality of disk array control units (1-1 , 1-2, 
1-3), each having one or more channel inter- 
face units (11) for interfacing with a host com- 

45 puter (50); one or more disk interface units (1 2) 

for interfacing with disk drives (5); a cache 
memory unit (14) for temporarily storing data 
which is written to or read out of said disk drives 
(5); and a shared memory unit (13) for storing 

50 control infonnatlon of data transfer between 

said channel interface unit (11) and disk inter- 
face unit (12) and said cache memory unit (14) 
and control infonnation for said disk drives (5), 
said disk array control unit Implementing the 

55 data read/write operation in response to a data 

read/write request from said host computer (50) 
by operating on said channel interface unit (11 ) 
to transfer data between said interface with said 
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host computer (50) and said cache memory unit 
(14) and operating on said disk interface unit 
(1 2) to transfer data between said disk drive (6) 
and said cache nnemory unit (14), 
said disk array control units (1-1, 1-2, 1-3) hav- 5 
ing, on the connection nodes of said channel 
interface units (1 1 ) and disk interface units (12) 
and said cache mennory units (14), a first con- 
nection path (25, 220) for the connection with 
other disk array control unit (1-1, 1-2, 1-3), and io 
having, on the connection nodes of said chan- 
nel interface units (11) and disk interface units 
(12) and said shared memory units (13), a sec- 
ond connection path (24, 210) for the connec- 
tion with the other disk array control unit (1 -1 , is 
1-2. 1-3); and 

a switch box (21 0) which is connected with said 
first and second connection paths of said disk 
array control units (12), thereby integrating said 
disk array control units (12) functionally into a 20 
single disk array controller. 

8. A method of increasing the number of disk array 
control units in a disk array controller which com- 
prises: 25 

a plurality of disk array control units each hav- 
ing one or more channel interface units for in- 
terfacing with a host computer; one or more 
disk interface units for interfacing with disk 30 
drives; a cache memory unit for temporarily 
storing data which is written to or read out of 
said disk drives; and a shared memory unit for 
storing control Information of data transfer be- 
tween said channel interface unit and disk in- 3s 
terface unit and said cache memory unit and 
control information for said disk drives, and im- 
plementing the data read/write operation in re- 
sponse to a data read/write request from said 
host computer by operating on said channel in- 40 
terface unit to transfer data between said Inter- 
face with said host computer and said cache 
memory unit and operating on said disk inter- 
face unit to transfer data between said disk 
drive and said cache memory unit, 45 
said disk array control units having, on the con- 
nection nodes of said channel interface units 
and disk interface units and said cache memory 
units, a first connection path for the connection 
with other disk array control unit, and having, so 
on the connection nodes of said channel inter- 
face units and disk interface units and said 
shared memory units, a second connection 
path for the connection with the other disk array 
control unit; and a switch box which is connect- ss 
ed with said first and second connection paths 
of said disk array control units, thereby integrat- 
ing said disk array control units functionally into 



a single disk array controller, 
said first and second connection paths and said 
switch box having connectors, 
said switch box having its connectors connect- 
ed by cables to connectors of the first and sec- 
ond connection paths of a disk array control unit 
to be added to said disk array controller. 

9. " A method of increasing disk array control units ac- 

cording to claim 8, wherein said switch box has an 
information table for storing Information indicative 
of the correspondence between the destination ad- 
dress of data which Is transferred through said 
switch box and information which indicates a disk 
array control unit to be connected, said information 
table being rendered the addition of correspond- 
ence information for a disk array control unit to be 
added to said disk array controller. 

10. A method of increasing disk array control units ac- 
cording to claim 8 , wherein said disk array control 
units are built on individual racks and said switch 
box is built together with a disk array control unit on 
one of said racks, said switch box having the con- 
nection of a plurality of cables which are connected 
to racks of other disk array control units. 

11 . A disk array controller comprising by being built on 
a single rack: 

a plurality of disk array control units (1-1, 1-2, 
1-3), each having one or more channel inter- 
face units (11) for interfacing with a host com- 
puter (50), one or more disk interface units (1 2) 
for interfacing with disk drives (5), a cache 
memory unit (14) for temporarily storing data 
which is written to or read out of said disk drives 
(5), and a shared memory unit (13) for storing 
control infonnation of data transfer between 
said channel interface unit (11) and disk inter- 
face unit (1 2) and said cache memory unit (14) 
and control infonnation for said disk drives (5), 
and implementing the data read/write operation 
in response to a data read/write request from 
said host computer (50) by operating on said 
channel interface unit (11) to transfer data be- 
tween said interface with said host computer 
(50) and said cache memory unit (14) and op- 
erating on said disk interface unit (1 2) to trans- 
fer data between said disk drive (5) and said 
cache memory unit (1 4), said disk array control 
units (1-1 , 1-2, 1-3) having, on the connection 
nodes of said channel interface units (11) and 
disk interface units (12) and said cache mem- 
ory units (14), a first connection path (25, 220) 
for the connection with other disk array control 
unit, and having, on the connection nodes of 
said channel interface units (11) and disk inter- 
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face units (12) and said shared memory units 
(13), a second connection path (24,210) forthe 
connection with the other disk array control 
unit; and 

a switch box (210) which is connected with said 5 
first and second connection paths of said disl< 
array control units, thereby integrating said disk 
array control units functionally into a single disk 
array controller 
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